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Abstract 


This  study  addrsssss  two  steps  in  a  process  known 
as  principle  components  analysis  using  Monte  Carlo 
techniques.  An  analysis  is  presented  of  two  popular 
dimensionality  assessment  techniques,  Kaiser's 
criterion  and  Catell’s  Scree  test.  The  factor 
interpretation  issue  is  addressed  through  a  regression 
study  in  which  the  grand  mean  square  error  between 


population  and  sample  factor  loading  matrices  is 
predicted.  The  notion  of  a  complexity  index  is  also 
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I  ntroduct i on 


Background 

Factor  analysis  is  a  widely  used  Multivariate  data 
analysis  technique.  This  technique  allows  an  analyst  to 
investigate  the  underlying  structure  of  a  set  of 
variables  over  which  data  has  been  gathered. 

Factor  analysis  has  seen  its  most  concentrated 
application  in  the  behavioral  sciences.  Even  though  the 
methods  and  models  of  factor  analysis  are  of  a 
statistical  nature,  factor  analysis  was  developed 
mainly  by  psychologists  ( Joreskog,  1979)  and  as  such  a 
literature  search  in  the  area  of  factor  analysis  will 
lead  one  to  such  journals  as  Psychometrika  and 
Psychological  Reports. 

Objectives  of  Factor  Analysis.  One  object  of 
factor  analysis  is  to  determine  the  underlying 
dimensionality  of  a  process  by  finding  independent 
factors  which  are  highly  related  to  one  or  more  of  the 
variables  in  question.  The  process  by  which  an 
investigator  determines  (via  factor  analysis)  the 
dimensionality  of  a  set  of  data  will  be  called,  for 
purposes  of  this  report,  the  dimensionality  assessment. 
After  the  dimensionality  assessment  has  been  made  it 
would  be  desirable  to  be  able  to  give  a  simple 
interpretation  to  each  of  the  factors  (McHichols,  1980) . 
At  this  point  a  process  know  as  rotation  is  used  in  an 
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attcapt  to  develop  a  simpler  more  easily  interpretable 
solution.  So,  another  abjective  for  the  investigator  is 
to  interpret  the  extracted  factors  correctly.  This 
abjective  will  be  called,  again  for  the  purposes  of 
this  report,  factor  interpretation. 

Principal  Component  Analysis.  This  report  will 
deal  exclusively  with  a  eethodology  known  as  principal 
components  analysis  (PCA) .  Psychologists  draw  the 
following  distinctions  between  factor  analysis  and 
principal  components  analysis.  In  factor  analysis  an 
attempt  is  made  to  find  a  certain  number  of  factors, 
fewer  than  the  number  of  variables,  such  that  the 
intercorrelations  between  the  variables  is  reproduced 
exactly.  In  PCA  , independent  factors  are  extracted  from 
the  data  until  a  sufficient  proportion  of  the  total 
variance  exhibited  by  the  data  is  reproduced  by  the 
factors.  Hence,  PCA  is  said  to  be  variance  oriented 
while1  factor  analysis  is  said  to  be  correlation 
oriented  (Joreskog, 1979) .  For  purposes  of  this  report 
PCA  will  be  considered  a  "factor  analytic"  procedure. 

Computational  Procedure.  The  PCA  computational 
procedure  consists  of  three  steps  (1)  the  preparation 
of  the  correlation  matrix,  <2)  the  extraction  of  the 
initial  factors  —  from  whence  a  dimensionality 
assessment  is  made  <this  step  is  where  the  investigator 
explores  the  possibility  of  data  reduction)  and  (3) 
rotation  to  a  tarm  ...  sc  at  ion  -  a  search  for  simple 
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and  interpretable  factors  <Niey197S>  (McNichols, 1980) . 

1.  Preparation  of  the  corral aton  matrix.  To 
prapara  tha  correlation  matrix  tha  fol lowing  procadura 
is  uaad.  First  let  n  obsevations  of  soma  random 
variable  be  given  by  a  nxl  vector 


Thus  »<e  have  n  observations  on  a  single  variable.  If  tw 
have  k  such  variables  we  can  assemble  them  in  a  matrix 
such  that  each  column  represents  n  observations  on  the 
k  variables.  This  data  matrix,  X,  is  a  nxk  matrix. 


XU  . xln 

x2i  : 

a  a 

•  a 

a  a 

a  a 

a  a 

xnl  . xnk 

a 


(Whara  tha  subscript  i  denotes  tha  ith  observation  and 
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the  subscript  j  denotes  the  jth  variable.) 

if  each  x. •  is  standardized  ie.  set  equal  to 

X  J 


(where  *x  -  is  the  sample  mean  of  the  jth  variable  and  s  . 

J  J 

is  sample  standard  deviation  of  the  jth  variable)  and 

T 

if  the  matrix  multiplication  X  X  is  performed  then  the 
resultant  product  is  the  sample  correlation  matrix. 

2.  Extraction  of  the  initial  factors.  The  second 

step  in  the  process  is  to  extract  the  eigenvalues  from 

T 

X  X.  The  extracted  eigenvalues,  when  rank  ordered,  are 
the  basis  for  most  of  the  currently  popular 
dimensionality  assessment  techniques.  Once  this 
assessment  has  been  made  the  eigenvectors  associated 
with  the  retained  eigenvalues  are  normalized,  these 
vectors  are  then  multiplied  by  the  square  roots  of 
their  respective  eigenvalues  and  assembled  in  matrix 
form.  This  matrix  is  called  the  factor  loadings  matrix, 
this  is  because  each  coefficient  can  be  said  to 
represent  a  "loading"  of  that  particular  variable  to 
the  factor. 

3.  Rotation  to  terminal  solution.  Since  the 
normalized  eigen vectors  (or  principal  components  as 
they  are  sometimes  called)  are  by  definition  mutually 
orthogonal,  the  PCA  methodology  has, therefore,  yields  a 
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basis  for  the  data  space  whose  di eens ion  is  less  than 
or  equal  to  the  rank  of  the  correlation  Matrix.  The 
loadings  Matrix  ,  or  factor  structure  Matrix,  is  not 
unique.  That  is  to  say,  the  factor  structure  Matrix  May 
be  rotated  arbitrarily  and  will  still  explain  the  sane 
anount  of  total  variance.  Rotation  schemes  have  been 
developed  to  "clean  up"  the  factor  structure  Matrix  and 
hence  aid  in  factor  interpretation  (McNichols, 1980) . 
This  rotation  for  interpretation  is  the  third  and  final 
step. 

For  a  More  detailed  Matheeatical  and  statistical 
development  see  (Lawley  and  liaxwel  1 , 1971) , 

(Harman, 1967) ,  (Harris, 1975) . 
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Problem  Statwnt 


One  Mjor  problem  facing  an  investigator  is  trying 
to  determine  how  many  factors  to  retain. 

Unfortunately,  the  dieensionality  assessment  procedure 
for  PCA  has  not  been  rigidly  defined.  Several 
alternatives  are  available  to  an  investigator.  These 
include  the  Scree  test  (catell,  1966),  Horn’s  test 
(Horn,  1965),  and  the  "fraction  of  variance  explained" 
test  (Kaiser,  1960).  There  does  not  appear  to  be 
evidence  that  any  one  of  these  is  superior  to  the 
others.  It  would  be  of  interest  to  determine  which 
test  is  the  most  powerful. 

Recent  research  suggests  that  factor  analytic 
procedures  can  be  significantly  influenced  by  sample 
size,  the  number  of  variables,  number  of  inherent 
factors,  complexity  of  the  inherent  factor  structure, 
and  the  interactions  between  these.  The  following 
questions  are  raised* 

1)  To  what  extent  is  the  dimensionality 
assessment  biased  by  these  influences?  How  is  the 
variance  of  the  dimensionality  assessment  affected? 

2)  How  is  factor  interpretation  affected  by 
these  influences?  More  specifically,  how  is  the  size 
of  the  mean  square  error  (MSE)  (or  perhaps  RMS  error) 
of  sample  loadings  affected?  Can  USE  (or  RHS  errors) 
be  reasonably  predicted  as  a  function  of  the 

for ament ioned  influences? 
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3)  An  investigator  beginning  a  factor  analytic 
study  starts  with  limited  information.  Assuming  he 
knows  the  correlation  matrix  of  the  sample  datai  is 
there  some  rule  of  thumb  based  on  the  condition  number 
of  this  matrix,  the  number  of  variables,  and  the  sample 
size  which  might  aid  him  in  estimating  errors  he  might 
expect  in  a  sample  factor  loadings  matrix? 
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Rcvisw  of  the  literature 

Drowne  (1968)  states, "There  is  no  statistical  test 
of  significance  for  the  number  of  factors  applicable  to 
the  principal  factor  estimates  and  rules  of  thumb  are 
generally  used  for  estimating  the  number  of  factors." 

It  is  important  to  note  that  factor  analysis  is 

not  devoid  of  statistical  inference  tools.  Cliff  and 

Hamburger  (1967)  assert  that 

"  The  results  available  from  statistical 
theory,  while  useful,  leave  a  large  area 
where  the  needs  of  the  investigator  are 
unsatisfied.  The  statistical  tests  for  the 
number  of  factors  are  all  tied,  naturally 
enough,  to  the  repective  methods  for 
estimating  factors.  Moreover  ,  these  methods 
of  estimation  are  either  computationally 
arduous  ....  or  unfamiliar  ....  Consequently, 
they  are  rarely  used  and  so  the  statistical 
tests  are  rarely  applied.  More  important  than 
this  is  the  fact  that  the  number  of  factors 
in  a  given  matrix  is  only  one  of  the  many 
concerns  of  the  investigator.  He  is 
interested  in  a  wide  variety  of  statistical 
questions,  and  he  is  interested  in  them  as 
they  arise  in  the  methods  of  factor  analysis 
currently  in  use. 

Factor  analysis  is  not  the  only  field  of 
statistics  where  questions  of  practical 
interest  have  been  too  complex  or  too 
difficult  to  specify  mathematically  for 
analytic  solution.  In  such  instances  it  is 
fairly  common  practice  to  take  the  Monte 
Carlo  approach,  in  which  samples  from  some 
specified  population  are  generated  by  some 
random  process. " 

Tucker  (1964)  and  Linn  (1968)  used  a  Monte  Carlo 
approach  to  study  what  Psychologists  term  pyschometri 
error.  Psychometric  error  arises  when,  say,  a  given 
battery  of  tests  are  measuring  factors  different  from 


those  -factors  it  was  designed  to  aaasura. 

Tucker’s  approach  entailed  the  perturbation  of 
idealized  coeeon  factor  loadings.  Tucker  constructed 
what  he  called  a  "foreal  model".  In  the  f areal  aodel , 
the  idealized  loadings  were  changed  by  randoa 
deviations.  Small  loadings  on  over  100  additional 
factors  were  also  generated  by  a  random  process.  In 
this  test,  then,  the  common  factor  loadings  were 
perturbed  and  this  effect  was  compounded  by  the 
presence  of  a  large  number  of  nuisance  factors.  This 
expanded  factor  loading  matrix  was  then  multiplied  by 
its  transpose  to  yield  a  correlation  matrix.  This 
correlation  matrix  was  then  factored  to  yield  a  factor 
loading  matrix.  This,  then,  is  the  factor  loading 
matrix  of  what  Tucker  calls  the  "simulation  model". 
Tucker  asserts  that  by  comparing  the  factor  loading 
matrix  of  the  "formal  model"  to  that  of  the  "simulation 
model"  one  can  measure  the  degree  of  psychometric  error 
bet wean  the  idealized  factor  structure  and  the 
perturbated  factor  structure. 

Horn  (1965)  suggested  a  novel  application  of  the 
Honte  Carlo  procedure  to  aid  in  dimensionality 
assessment.  Horn’s  procedure  is  as  follows* 

1)  Given  a  data  matrix  of  m  measurements  on  n 
variables,  the  sample  correlation  matrix  is  prepared 
and  its  eigenvalues  are  extracted. 

2)  Generate  n  samples  of  m  independent 


observations  from  a  normally  distributed  population  of 
random  numbers.  Prepare  this  correlation  matrix  and 
extract  its  eigenvalues.  This  process  is  to  be  repeated 
K  times  to  yield  K  sets  of  eigenvalues.  Each  set  of 
eigenvalues  is  rank  ordered  and  averages  of  the  largest 
eigenvalue,  second  largest  eigenvalue,  etc.  are 
computed  across  the  K  sets.  (Note  that  K  should  be 
chosen  large  enough  to  ensure  a  good  estimate  for  the 
means) . 

3)  Compare,  in  rank  order,  the  eigenvalues  due 
to  the  real  world  data  with  the  eigenvalues  of  the  K 
randomly  generated  correlation  matrices.  Pick  the 
smallest  eigenvalue  from  the  real  world  data 
correlation  matrix  whose  value  is  larger  than  its 
counterpart  from  the  randomly  generated  matrices. 

Horn  argues  that  this  procedure  will  reduce  the 
number  of  factors  that  would  have  been  retained  by, 
say,  the  Guttman  (1954)  weaker  lower  bound.  (Guttman’s 
weaker  lower  bound  is  more  commonly  referred  to  as 
Kaiser's  criterion.  Kaiser  (1960)  first  adapted  a 
procedure  of  retaining  all  factors  whose  eigenvalues 
were  all  greater  than  or  equal  to  one.).  Horn  feels 
that  too  many  factors  will  have  been  retained  by  the 
other  criteria  due  to  the  fact  that  these  tests  ignore 
sampling  error  and  the  error  which  Horn  refers  to  as 
least  squares  "capital ization".  Least  squares 
"capi tal ization"  refers  to  the  fact  that,  in  factor 
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analysis,  ths  first  derived  factor  is  constructed  in 
such  a  fashion  as  to  take  up  as  such  of  the  variance 
present  as  possible  (in  the  least  squares  sense)  and  as 
such  "capitalizes"  on  the  chance  fluctuations  in  a 
particular  set  of  data.  Horn,  then  (to  use  engineering 
vernacular),  claims  that  the  "real  "  eigenvalues  ride 
above  those  eigenvalues  due  to  the  inherent  "noise"  of 
the  process  being  investigated.  Horn  makes  no 

pretensions,  however  ,  as  to  the  validity  of  his 
procedure  and  essentially  presents  his  rationale  as  a 
rule  of  "thumb".  Although  Horn's  reasoning  appealed  to 
this  author,  experts  in  the  field  have  not  received  the 
rationale  with  open  arms.  Cliff  and  Hamburger  (1967)  go 
so  far  as  to  state  that  Horn's  arguments  are  purely 
"verbal"  and  present  a  hypothetical  counter  example. 

The  counter  example  goes  as  follows:  suppose  the 
experimental  situation  is  such  that  there  is  a  very 
large  common  factor  present  and  a  second  much  smaller 
one.  The  eigenvalue  of  the  large  factor  may  be  so  large 
that  all  succeeding  eigenvalues  are  much  less  in 
magnitude  than  one.  In  the  random  data  we  should  find 
several  eigenvalues  greater  than  one  (approximately  1/2 
of  them).  Hence,  when  the  eigenvalues  comparisions  are 
made  we  will  only  retain  the  single  large  factor.  So, 
in  this  counter  example,  Horn's  rationale  has 
underestimated  the  number  of  true  factors.  In  defense 
of  Horn,  however,  one  might  wonder  how  important  that 
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other  saall  factor  was  to  the  analysis,  that  is  to  say, 
what  is  the  penalty  of  ignoring  such  saall  factors? 

Browns  (1968)  published  a  lengthy  Monte  Carlo 
study  in  which  he  exaained  the  effects  that  increasing 
saeple  size,  and  increasing  the  number  of  variables 
while  holding  constant  the  number  of  factors  had  on 
estimates  of  factor  loadings.  Browne  compared  and 
contrasted  several  factor  analytic  techniques  (PCA  was 
not  analyzed,  but  a  hybrid  technique  which  placed  the 
communal i t i es  on  the  diagonal  of  the  correlation  matrix 
was  examined).  The  same  papulation  factor  matrix  was 
used  throughout  his  study.  This  matrix  had  16  variables 
which  loaded  on  4  factors  with  communal i ties  ranging 
from  .9  to  .1.  Using  a  method  due  to  Odel  and  Feiveson 
(1966),  random  correlation  matrices  were  generated  and 
factor  analyzed  using  five  different  techniques.  Browne 
presents  extensive  tables  which  show  various  attributes 
of  the  sample  loadings  he  obtained.  Browne  demonstrates 
the  superiority  of  of  a  technique  called  "maximum 
likelihood”  to  estimate  factor  loadings.  He  also 
studied  the  dimensionality  assessment  problem  and  found 
that  although  none  of  the  methods  proved  completely 
satisfactory,  the  decision  procedure  based  on  a 
sequence  of  likelihood  ratio  tests  and  the  criterion  of 
number  of  eigenvalues  greater  than  one  of  the  sample 
correlation  matrix  gave  the  best  results  of  the  methods 


considered.  The  other  criterion  considered  was  due  to 


Saundars  (I960),  this  technique  involves  taking,  at 
each  iteration  of  a  factor  analytic  technique  called 
Thomson's  aethod  (Thoason, 1934) ,  the  nuaber  of  positive 
eigenvalues  greater  than  the  absolute  value  of  the 
seal lest  eigenvalue  as  a  criterion  for  the  nuaber  of 
factors  to  be  used  for  the  following  iteration. 

Linn  (1968)  suggested  an  approach  to  the 
diaensionality  assessment  problem  which  is  similar  in 
spirit  to  Horn’s  procedure.  Instead  of  analyzing  the 
real  variables  and  the  randomly  generated  variables 
separately, they  are  analyzed  jointly.  In  Linn’s 
procedure  k  new  variables  are  introduced  to  the  real 
data  and  if  there  are  m  observations  on  each  of  the 
original  variables,  m  independent  observations  are 
generated  for  the  k  new  variables.  All  the  observations 
across  both  sets  of  the  variables  are  then  used  to 
prepare  a  correlation  matrix  which  is  subsequently 
factor  analyzed.  The  dimensionality  assessment  is  then 
based  on  those  factors  which  are  primarily  composed  of 
real  variables.  The  factors,  whose  main  loadings  are 
those  of  the  generated  variables,  are  to  be  discarded, 
then,  as  random  factors.  Linn  (1968)  expanded  the  scope 
of  his  original  1964  study.  Linn’s  1964  study  was 
limited  by  the  fact  that  only  two  observed  matrices 
were  used  and  each  of  these  was  based  on  a  sample  of 
size  80. 

Catell  (1966)  suggests  a  brief,  easy  to  apply  test 
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for  dimensionality  assessment.  The  test  entails  simply 
graphing  the  eigenvalue  magnitudes  versus  the  factor 
number  (the  factor  number,  for  instance,  of  the  factor 
corresponding  to  the  largest  eigenvalue  is  lj  the 
second  largest  eigenvalue  is  2,  etc.).  Catell  noticed 
that  often  in  empirical  data  a  "break"  was  exhibited  in 
the  eigenvalue  curves.  This  break,  Catell  reasoned, 
signalled  the  the  beginning  of  the  trivial  factors  and 
started  a  more  gradually  sloping  linear  trend  in  the 
eigenvalue  curve  which  resembled  a  scree  line.  A  scree 
line  is  the  shape  that  rocks  sliding  off  a  hill  will 
assume  at  the  bottom  of  the  hill.  Although  this  test  is 
not  Monte  Carlo  in  nature  it  is  mentioned  because  Horn 
(1969)  and  Linn  (1968)  both  noticed  this  break  in 
eigenvalues  magnitude. 

Hamburger  (Cliff  and  Hamburger , 1967)  performed  a 
limited  Monte  Carlo  study  to  examine  the  apparent  break 
in  eigenvalue  curves.  To  be  more  specific.  Hamburger 
sought  to  find  a  break  where  two  adjacently  ranked 
eigenvalues  are  sharply  different  in  size  while  on  both 
sides  of  the  break  the  decrease  is  more  gradual.  He 
reports  that  when  sample  sizes  as  are  high  as  400,  then 
using  the  break  in  eigenvalue  magnitude  as  a  decision 
rule  for  factor  retention  was  flawless  (at  least  for 
the  matrices  examined  in  the  study)  and  when  sample 
sizes  were  reduced  to  100  the  rule  usually  gave  correct 
results.  Apparently  4  different  simple  structure  types 


were  examined  over  some  160  randomly  generated  sample 
correlation  matrices.  Hamburger  fails  to  report  the 
number  of  population  common  factors  and  variables  that 
mere  used  to  generate  his  sample  correlation  matrices 
and  as  he  points  out,  "These  conclusions  are  of  course 
tempered  by  the  fact  that  results  probably  depend  on 
the  number  and  size  of  common  factors  and  the  number  of 
variables 

Joreskog  (1963)  studied  the  sampling  errors  of 
individual  loadings  on  unrotated  common  factors.  He 
used  a  factor  analytic  technique  which  he  developed  in 
the  1962-1963  time  period  (Joreskog, 1962, 1963)  .  this 
procedure  is  reported  to  yield  factors  very  similiar  in 
appearance  to  those  generated  by  the  PCA  procedure.  In 
this  particular  study  Joreskog  generated  a  small  number 
of  sample  correlation  matrices,  under  various 
conditions,  and  analyzed  various  characteristics  of 
them.  Two  cases  stand  out  in  particular.  In  one  case, 
six  20— variable  sample  correlation  matrices  were 
generated  over  uncorrelated  factors.  In  three  of  the 
matrices,  the  factor  scores  were  assumed  to  be  normally 
distributed  while  in  the  other  three  matrices  the 
factor  scores  were  assumed  to  have  a  rather  strong 
skewness.  Sample  sizes  for  each  set  of  three 
correlation  matrices  were  100,  200,  and  300.  When  the 
sample  correlation  matrices  were  factor  analyzed  to 
yield  sample  factor  loading  matrices  it  was  found  that 
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the  root  Man  square  (res)  deviation  of  saeple  loadings 
to  population  loadings  was  soeewhat  less  than  1  / /fi, 
where  N  is  saeple  size,  the  approximate  standard  error 
of  a  zero  correlation.  There  were  no  consistent 
differences  reported  due  to  skewness. 

A  second  case  showed  a  sharp  contrast  to  the  first 
case.  The  population  factor  structure  chosen  for 
examination,  in  this  case,  was  a  lO— variable  three 
factor  structure.  The  structure  exhibited  perfect 
simple  structure.  Joreskog  generated  10  sample 
correlation  matrices  for  each  of  the  three  sample 
sizes.  This  time  the  differences  between  the 
population  loadings  and  the  sample  loadings  were  very 
great.  In  some  samples  one  or  more  of  the  factors 
generated  could  not  be  confidently  matched  with 
population  factors.  These  errors  decreased  slightly  as 
the  sample  size  was  increased. 

Joreskog  next  decided  to  rotate  the  previous  set 
of  factors,  via  a  least  squares  procedure,  to  its 
population  structure.  The  resultant  standard  errors  of 
the  rotated  loadings  were  quite  small)  somewhat  smaller 
than  1//N.  Also,  the  sampling  errors  of  non— zero 
loadings  tended  to  be  smaller  than  those  for  zero 
loadings,  in  the  same  manner  that  sampling  errors  for 
the  Pearson  correlation  coefficients  are  proportional 
to  1-r*.  Browne  (1968)  also  observed  that  the  sampling 
errors  of  rotated  factor  loadings  were  about  the  same 
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as  correlation  coefficients,  of  the  order  1//n. 


Joreskog  also  looked  at  the  sampling  error  of 
rotated  loadings  when  the  factor  scores  were  given  from 
a  rectangular  distribution.  He  found  that  although  the 
standard  errors  of  the  loadings  rose  slightly  in  this 
case  they  remained  less  than  1//R  and  exhibited  the 
same  proportional i ty  to  the  magnitude  of  the  original 
loadings  as  found  in  the  normally  distributed  factor 
score  cases.  This  result  (coupled  with  the  skewed 
factor  score  distribution  results  previously  mentioned) 
suggest  that  factor  analytic  methods  are  reasonably 
robust  in  respect  to  moderate  departures  from  the 
normally  distributed  factor  score  assumption. 

Hamburger  (Cliff  and  Hamburger,  1967)  observed  a 
bias  in  the  estimates  of  individual  factor  loadings. 
Hamburger  does  not  state  his  original  factor  structure 
but  he  does  state  that  the  sample  sizes  studied  were 
100  and  400.  He  noticed  that  larger  loadings  (.6  to  .9) 
were  consistently  underestimated  while  the  smaller 
loadings  (.2  to  .5)  showed  a  tendency  to  be 
overestimated.  Browne's  (1968)  data  exhibited  a 
similiar  trend.  The  factor  analytic  procedure  used  for 
these  cases  was  PCA  with  squared  multiple  correlations 
placed  on  the  diagonal  of  the  input  correlation  natrix. 

Cliff  and  Pennell  (1967)  studied  in  some  detail 
the  effects  of  communal ity,  factor  strength,  and 
loading  size  on  the  sampling  characteristics  of  factor 
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loadings.  The  sampling  characteristics  addressed  in 
this  study  were  referred  to  as  “stability*9  and  "bias". 
Stability  refers  to  the  amount  of  variability  an 
individual  factor  loading  might  be  expected  to  exhibit 
from  sample  to  sample.  Bias  refers  to  the  extent  to 
which  the  mean  of  a  sampling  distribution  of  factor 
loadings  might  be  expected  to  approximate  the 
popul at i on  1 oad i ng . 

Two  model  population  factor  structures  were 
studied.  Each  factor  structure  was  constructed  with 
four  different  factor  strengths,  four  different 
communal i ti es,  and  four  different  loading  sizes.  The 
loadings  were  situated  in  such  a  fashion  to  facilitate 
various  comparisons.  For  instance,  by  choosing  selected 
loadings,  it  was  possible  to  compare  the  effect  of 
factor  strength  on  given  loadings  of  equal  magnitude 
and  communa.1  i ty.  The  loadings  of  the  first  model 
structure  ranged  from  .9  to  .45,  while  the  second’s 
loadings  ranged  from  .7  to  .35.  Fifty  sample 
correlation  matrices  were  generated  for  each  of  the 
model  structures.  The  sample  sizes  were  not  given. 

These  matrices  were  factor  analyzed  by  the  PCA 
procedure  with  communal! ties  placed  on  the  diagonal. 
Four  factors  were  extracted  and  the  resultant  structure 
was  rotated,  via  a  least  squares  procedure  due  to  Cliff 
(1966),  to  fit  the  model  structure.  The  means  and 
standard  deviations  of  the  Individual  factors  were 
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calculated  over  the  50  sample  structures.  Individual 
-factor  loadings  exhibited  a  wide  variety  of  saaple 
frequency  distributions. 

Several  interesting  influences  affecting  stability 
were  noted.  A  non-zero  loading  which  was  associated 
with  a  large  communality  was  almost  always  associated 
with  smaller  standard  deviations  then  those  loadings 
associated  with  smaller  communal i ties.  A  si mi liar  trend 
was  noticed  for  zero  loadings.  It  was  also  noticed  that 
the  larger  a  loading  was,  the  smaller  its  standard 
deviation  tended  to  be.  The  trends  were  presented  quite 
clearly  in  graphs  in  the  text  of  the  article.  The 
authors  further  noticed  that  stronger  factors  tend  to 
produce  more  stable  loadingsy  apparently  independent  of 
the  other  parameters. 

Cliff  and  Pennel  next  summarized  these 
observations  in  a  multiple  regression  study.  The 
dependent  variable  was  the  standard  deviation  of  the 
factor  loadings.  There  were  7  independent  variablesit 

1.  Size  of  population  factor  loading 

2.  Population  communality  of  the  variable 

3.  Number  of  non-zero  loadings  on  the  factor 

4.  Number  of  non-zero  loading  on  the  variable 

5.  Squared  loading  on  the  factor 

6.  Discrepancy  between  loading  and  other  loadings 
on  the  factor 

7.  Mean  loading  on  the  factor 

The  correlation  matrix  was  prepared  and 
correlations  of  the  order  .80  were  noted  for  predictors 
1,2,S,7.  Using  a  stepwise  regression  routine  it  was 
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found  that  six  of  tha  variables  could  account  for 

nearly  B9X  of  the  variance  present.  The  best  two 

predictors  were  6  and  7  which  together  explained  86%  of 

the  variance.  The  authors,  however,  point  that 

predictor  pairs  2  and  9  or  2  and  7  give  r*s  of  .842  and 

.843  respectively.  No  power  transforeations  or 

interactions  were  tested.  The  authors  also  report 

instances  of  bias  in  the  saeple  factor  loading  eatrix 

but  do  not  attempt  to  characterize  it. 

Cliff  and  Pennell  summarize  by  concluding 

"...that  communal ity  rather  than  loading  size 
is  the  important  determiner  of 
stabi 1 ity. . .Higher  communal i ties  mean  not 
only  greater  stability  for  the  loadings  of 
specific  tests  (variables)  but  also  load  to 
stronger  factors  which  mean  that  the 
stability  of  all  the  loadings  is  improved." 

Cliff  and  Pennell  did  not  address  sample  size  and 
its  possible  interaction  with  other  effects.  Also  the 
number  of  variables  and  factors  were  not  varied. 

Pennell  (1968)  extended  the  study  by  looking  at 
the  influences  of  communal  ity  and  N  (N  is  the  sample 
size  used  to  generate  sample  correlation  coeff icients) 
on  the  sampling  distributions  of  factor  loadings. 
Pennell  inserted  variables  with  different  communalities 
in  randomly  constructed  factor  structures.  Samples  were 
drawn  from  the  population  correlation  matrix  to  prepare 
a  sample  correlation  matrix.  The  sample  correlation 
matrix  was  then  factor  analyzed  using  PCA  with  squared 

multiple  correlations  on  the  diagonals.  As  in  Cliff  and 
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Pcnnil 1 ’ s  1967  study,  the  standard  deviation  of  the 
factor  loading  was  taken  to  be  the  dependent  variable. 

Pennel,l  noticed  in  the  1967  study  that  uni  vocal 
variables  (variables  which  load  on  a  single  factor)  not 
only  facilitated  rotation  and  subsequent  factor 
interpretation  but  also  resulted  in  smaller  sampling 
errors.  Further  it  was  noticed  that  zero  loadings  seem 
to  exhibit  a  greater  degree  of  variability  than  a 
non-zero  loadings.  Pennell  hypothesised  that  the 
variability  of  a  loading  might  increase  with  its 
complexity  across  the  factors.  To  avoid  the  confounding 
of  error  in  complex  variables  Pennell  used  only 
uni vocal  tests  inserted  in  randomly  constructed  factor 
structures. 

The  research  design  chosen  was  a  two  way  analysis 
of  variance  with  5  levels  of  the  two  independent 
variables,  N  and  communal ity.  The  levels  of  N  were 
taken  as  100,  ISO,  300,  600,  and  2500.  The  levels  of 
communal  ity  were  taken  as  .1,  .3,  .5,  .7  and  .9.  for 
the  test  variables.  The  test  variables  were  inserted 
into  a  randomly  constructed,  12  variable  by  2  factor, 
population  structure  (The  test  variable’s  position  in 
the  structure  was  also  determined  in  a  random  fashion.) 
Three  such  random  structures  were  generated  for  each  of 
the  ANOVA's  25  cells.  These  75  structures  then  were 
used  to  generate  100  sample  correlation  matrices.  These 
correlation  matrices  were  then  factor  analyzed  in  the 


fashion  previously  mentioned  and  rotated  via  Cliff's 
<1966)  procedure  back  to  the  population  factor 
structure.  The  standard  deviations  of  the  zero  and 
non— zero  sample  correlation  matrices  were  calculated 
for  both  blocks  of  correlation  matrices.  Hence  each 
cell  of  the  ANOVA  contained  three  replications  of 
sample  standard  deviations.  The  subsequent  fixed 
effects,  two  way  ANOVA  revealed  that  both  main  effects 
and  their  interactions  were  significant.  This  was  true 
for  both  non— zero  and  zero  loadings.  It  was  noticed 
that  while  the  F  ratio  for  N  remained  approximately  the 
same  for  both  non-zero  and  zero  loadings,  the  F  ratio 
due  to  communal ity  was  strongest  for  non-zero  loadings. 

Graphs  were  drawn  that  depicted  the  standard 
deviations  Inan-zera  loadings)  of  the  various  test 
variables  as  functions  of  1/</n.  The  results  were 
striking,  in  all  but  one  case,  clear  linear  trends  wore 
observed.  It  was  also  clear  from  the  graphs  that  the 
standard  deviations  were  conditional  on  the  magnitude 
of  the  variable’s  communal ity.  Similar  trends  were 
observed  for  zero  loadings  although  differences  due  to 
communal i ties  were  harder  to  discern. 

Pennell  asserts  that  his  study  has  demonstrated 
clearly  the  advantage  of  developing  factor  pure 
(univocal)  variables  for  studies  of  psychological 
traits,  because  these  variables  exhibit  the  smallest 
amount  of  sampling  errors  in  the  non-zero  loadings  (the 
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loading  usually  of  Most  interest. ) 

Pennel  presents  tables  which  define  93X  confidence 
intervals  about  the  zero  loadings  as  a  function  of  N. 
These  tables  graphically  refute  the  rule  of  thumb  that 
selects  only  loadings  greater  than  .3  as  being 
significant.  This  author  reminds  the  reader  that  a 
"forced  fit"  rotation  scheme  was  employed  to  generate 
these  values  and  as  such  the  values  in  the  table  arise 
when  the  correct  structure  is  known,  a  priori,  and  the 
sample  loading  structure  is  rotated  to  fit  it.  Hence 
errors  here  are  due  to  the  factor  analytic  method  and 
sampling  errors.  Obviously,  investigators  may  imploy 
factor  analytic  and  rotation  schemes  which  will  produce 
effects  other  than  those  due  only  to  sampling  error. 
Pennell  closes  his  paper  with  an  interesting  table  in 
which  he  shows  the  size  of  sample  loadings  necessary  to 
be  signif icately  different  from  the  non-zero  loadings 
he  tested  (.9,  .8,  .7,  .6,  .5)  at  an  alpha  of  .03.  For 
example,  when  N  is  100,  a  sample  loading  would  have  to 
be  less  than  .79  to  reject  the  hypothesis  that  it  was 
actually  .90. 

Manners  and  Brush  (1979)  studied  the  "reliability” 
of  factor  analytic  techniques.  Reliability  is  defined 
as  (a)  the  mean  squared  error  between  factor  loadings 
for  sample  and  population  factor  loading  structures  and 
(b)  the  ability  of  a  factor  analytic  model  to  capture 
correctly  the  number  of  factors  in  the  population.  The 


rncarch  endeavored  to  compare  the  reliability  of  four 
separate  factor  analytic  techniques  with  respect  to  the 
effects  of  sample  size,  number  of  variables,  and  number 
of  factors.  An  analysis  of  variance  approach  was  used 
where  the  treatments  were  taken  as  a)  the  number  of 
variables,  b)  the  number  of  observations  (observations 
on  the  variables),  and  c)  the  four  different  factor 
analytic  models.  All  possible  interactions  were  also 
examined. 

The  experimental  procedure  called  for  use  of  a 
factor  structure  due  to  Browne  (1968).  This  structure 
was  divided  into  two  experimental  conditions;  the  first 
was  16  variables  and  4  factors  and  the  second  was  12 
variables  with  the  same  4  factors.  (The  reader  should 
note  that  factors  II  and  III  in  Browne's  structure  are 
not  orthogonal.  The  angle  between  these  factors  is 
aproximately  77  degrees.  Since  only  an  orthogonal 
rotation  was  used  in  the  analysis,  one  wonders  why 
Browne  did  not  employ  only  mutually  orthogonal 
factors.)  Ten  random  correlation  matrices  were 
generated  from  each  of  the  experimental  population 
factor  loading  structures  for  sample  observation  sizes 
of  N*100  and  N«500,  respectively.  Hence  10  times  2 
times  2  *  40  sample  correlation  matrices  were  factor 
analyzed  by  the  four  differnent  techniques.  The  four 
different  techniques  included  PCA  with  initial 
communal ity  estimates  placed  on  the  diagonal. 


Tbrn  dimensionality  assessment  ruin  wars  tntad 
for  the  hybrid  PCA  technique.  The  first  two  ruiaa  were 
to  choosa  tha  nuaber  of  factors  associated  with  the 
eigenvalues  of  eagnitude  greater  than  1  and  0, 
respectively.  Saunder’s  method  which  was  mentioned 
earlier  in  the  literature  review,  was  also  tested.  For 
the  eigenvalues  greater  that  1  rule,  28%  of  the 
dimensionality  assessments  were  correct.  The  remaining 
assessments  were  within  2  factors  of  being  correct  with 
35%  predicting  3  factors  and  37X  predicting  2  factors. 
The  eigenvalues  greater  than  0  rule  was  within  one 
factor  for  45%  of  the  assessments  and  high  for  the 
rest,  and  the  variance  was  much  larger  for  this  rule 
than  the  eigenvalues  greater  than  1  rule. 

In  the  fixed  effects  ANOVA  experiment  each  cell 
contained  10  replications  of  mean  square  loading  errors 
(10  correlation  matrices  for  each  cell;  4  (models) 
times  2  (sample  sizes)  times  2  (#  of  variables)  «  16 
cells).  All  the  treatments  and  treatment  interactions 
proved  significant  (alpha=.05),  save  the  interaction 
between  number  of  variables  and  number  of  observations. 

To  summarize;  Manners  and  Bush  provide  evidence 
that  factor  analytic  reliability  is  influenced  by 

1)  The  specific  factor  analytic  models  chosen. 

2)  The  interaction  of  factor  analytic  model 
choice  and  number  of  variables. 

3)  The  interaction  of  factor  analytic  model 
choice  and  sample  size. 

4)  The  interaction  of  factor  analytic  model 
choice,  sample  si2e  and  number  of  variables. 
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In  passing,  on*  also  not*s  that  all  th*  aain 
treatments*  variables,  observations,  and  models  Mere 
significant.  Sampling  error  decreased  as  observations 
increased.  Sampling  error  also  decreased  as  the  number 
of  variables  increased,  much  as  is  observed  in  multiple 
regresion  analysis. 

This  ends  the  literature  review  section  dealing 
with  Monte  Carlo  experimental  work  in  factor  analysis. 
The  following  paragraphs  present  a  brief  overview  of 
recent  literature  which  is  related  either  to  the/ 
research  in  this  report  or  the  factor  analysis  problem 
in  general . 

The  rotational  scheme  used  in  this  report  is  due 
to  Schoneman  (1966).  This  rotational  procedure  allows 
one  to  rotate  a  sample  factor  loadings  matrix  to  given 
target  matrix  (usually  the  hypothesized  population 
factor  loadings  matrix).  The  rotation  is  accomplished 
in  such  a  manner  as  to  minimize,  in  a  least  squares 
sense,  the  residual  differences  between  the  rotated 
Mtrix  and  its  target. 

Odell  and  Feiveson  (1966)  provide  the  methodology 
by  which  all  the  reviewed  studies  generated  sample 
values  from  multinormal  populations  with  given 
covariance  structures.  An  algorithm  for  the  bivariate 
normal  is  given  by  Naylor  (1966). 

A  mathematical  entity  known  as  the  condition 
number  of  a  matrix  is  used  in  this  report.  The 
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condition  number  of  a  matrix  is  an  especially  useful 
tool  in  systems  of  linear  equations.  If  small  errors  in 
the  right-hand  side  or  coefficients  of  a  linear  system 
produce  a  large  effect  on  the  solution,  then  the  system 
of  equations  is  said  to  be  ill-conditioned.  The 
condition  number  of  a  matrix  serves  as  an  index  of  this 
ill— conditioning.  The  condition  number  of  a  matrix  is 
given  by  the  absolute  value  of  the  ratio  of  that 
matrix’s  largest  eigenvalue  to  its  smallest.  Westlake 
(1968)  offers  a  clearly  written  text  on  the  application 
of  the  condition  number  and  other  measures  as  applied 
to  matrix  inversion  and  linear  equations.  Belsley,  et 
al .  (I960)  discuss  the  effects  of  ill  conditioning  by 

applying  the  condition  number  in  detecting  collinearity 
in  multiple  regression. 

Anderson  (1958)  is  recommended  for  rigorous  yet 
succinct  theoretical  treatment  of  PC  A.  Joreskog  (1979) 
offers  a  novel  introduction  to  factor  analysis  and  its 
associated  vernacular  using  the  concept  of  partial 
correlations  as  a  starting  point.  The  second  paper  in 
the  book  deals  with  statistical  tests  for  confirmatory 
factor  analysis,  the  only  such  statistical  tests  this 
author  was  able  to  find.  Also  there  is  an  interesting 
article  by  Catell  and  Sullivan  <1962)  in  which  the 
concepts  of  factor  analysis  are  made  clearer  to  the 
novice  through  the  use  of  a  physical  example  using  cups 
of  coffee. 
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Objectives  of  the  Research 

The  objectives  o-f  this  research  effort  were  the 
■followings 

1)  Develop  software  which  will  allow  a  user  to 
study  the  influences  of  sample  size,  number  of 
variables,  number  of  factors,  and  complexity  of  the 
factor  structure  in  PCA.  This  software  was  to  allow  a 
user  to  input  either  a  particular  structure  or  given 
covariance  matrix. 

2)  Address  the  three  questions  raised  in  the 
problem  statement. 

3)  Summarize  recent  developments  in  this  area  as 
found  during  a  literature  search. 
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Scope  of  the  Research 

DAeensionalvty  Assessment.  Kaiser's  criterion  and 
the  Scree  test  are  the  two  dieensionality  assessment 
procedures  to  be  addressed  in  this  report.  These  two 
procedures  appear  to  be  the  most  popular.  It  was  also 
felt  that  testing  Horn's  procedure  on  the  CDC  6000 
would  prove  cost  prohibitive. 

Factor  Interpretation.  This  report  did  not  treat 
the  problem  of  selecting  the  most  appropriate 
rotational  technique.  This  report  assumes  that  the 
correct  rotational  scheme  is  applied.  This  report 
attempts  to  provide  a  tool  by  which  investigators  can 
estimate  the  magnitude  of  factor  loading  errors  to  be 
incurred  under  various  experimental  conditions.  In 
particular,  sample  sizes  were  taken  from  the  range 
10—100,  the  number  of  variables  ranged  10-15,  the 
number  of  factors  ranged  from  3-6.  In  total  27  separate 
population  structures  were  examined.  Each  structure  was 
examined  at  sample  sizes  of  10,  25,  50,  100. 
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Approach  to  the  Research 

The  research  design  used  in  this  report  was  as 
follows*  Initially,  seven  10  variables  by  3  factors 
structures  were  examined.  These  structures  will  be 
referred  to  as  the  "original"  structures.  The  original 
structures  are  given  in  figure  1.  These  original 
structures  are  the  lower  left  hand  point  in  the  graph 
of  the  research  design  given  in  figure  2,  with 
coordinate  (3,10).  Next,  four  of  the  original 
structures  were  selected  to  be  perturbed  by  the 
presence  of  added  nuisance  variables.  These  structures 
are  given  in  figure  3.  Five  and  two  nuisance  variables 
were  added  to  these  structures, respectively.  The 
nuisance  variables  were  chosen  to  load  on  single 
factors  and  have  communal i ties  between  .01  and  .09. 
These  eight  structures  are  coordinates  (3,15)  and 
(3,12)  on  the  research  design  graph.  The  coordinates 
(4,10)  and  (6,10)  consist  of  three  structures  each, 
perturbed  by  the  presence  of  one  and  three  nuisance 
factors.  The  nuisance  factors  were  constructed  such 
that  all  factors  were  orthogonal  (save  factors  fJ  and  VI 
in  structure  17,  they  form  an  angle  of  approximately  74 
degrees,  as  it  turned  out  this  discrepancy  was 
insignificant,  as  in  Browne’s  (1968)  structure) .  The* 
structures  are  given  in  figure  4.  To  complete  the 
design  the  same  three  structures  were  perturbed  by  both 
nuisance  variables  and  factors.  These  structures  are 
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Figure  1. 


Original  Factor  Structures 
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Figure  3.  "Perturbed  Factor  Structures i  Nuisance  Variables 
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Figure  4.  Perturbed  Factor  Structures* 
Nuisance  Factors 
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Figure  5.  Perturbed  Factor  Structures! 
Nuisance  Factors  and  Variables 
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given  in  -figure  5.  These  structures  are  coordinates 
(4,12)  and  (6,15)  in  the  research  design  graph. 

Each  o-f  the  27  structures  were  examined  over 
sample  sizes  of  10,25,50,  and  lOO.  These  sample  sizes 
are  considerablely  lower  than  those  used  in  the 
reviewed  studies:  100  and  400  were  used  by  Hamburger 
(1967),  Joreskog  (1963)  used  100,200,  and  300,  Cliff 
and  Pennell  (1968)  used  100,150,300,600,  and  2500, 
while  Manners  and  Brush  (1979)  used  100  and  500.  These 
studies  reported  that  experimental  results  were,  in 
general,  only  moderately  improved  by  increasing  sample 
size.  Therefore,  it  was  thought  that  the  lower  sample 
sizes  would  provide  not  only  more  interesting  results 
but  shed  some  light  on  just  how  many  samples  are 
required  to  perform  an  accurate  PCA  in  the  given 
experimental  region. 

Each  structure  and  sample  size  combination  was 
analyzed  according  the  following  experimental 
procedure: 

1)  A  population  covariance  matrix  was  formed  by 
multiplying  the  population  structure  matrix  by  its 
transpose. 

2)  The  appropriate  number  of  sample  vectors  were 
drawn  randomly  from  the  population  covariance  matrix. 

3)  The  sample  vectors  were  then  used  to  form  a 
sample  correlation  matrix.  The  condition  number  of 
this  matrix  was  calculated  at  this  step. 
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4)  The  sample  correlation  eatrix  was  then  factor 
analyzed  by  the  PCA  procedure.  The  correct  number  of 
factors  were  retained.  Dimensionality  assessment 
statistics  were  collected  at  this  step. 

9)  A  factor  loadings  matrix  was  prepared  and 
rotated  via  a  least  squares  procedure  due  to  Schoneman 
(1966)  back  to  the  original  population  structure.  The 
mean  square  deviation  of  sample  loadings  from 
population  loadings  was  calculated  at  this  step. 

6)  Steps  2-5  were  repeated  1000  times  for  each 
structure-sampl e  size  combination. 


Complications  in  a  Population  Structure 

Rationale.  One  difficulty  in  a  study  such  as  this 
is  in  determining  what  structures  should  be  examined. 
The  limited  experience  of  this  author  indicates  that 
"simple"  structures,  whose  variables  load  on  no  more 
than  a  few  factors  and  which  contain  many  zeros,  are 
probably  of  the  greatest  use.  This  intuitive  feel  is 
merely  a  vague  generalization  of  Thurstones  criteria 
(see  Harman,  1967,  pg.  98)  which  is  widely  accepted  as 
a  desirable  quality  of  a  population  structure.  In  any 
event,  more  "complicated'*  structures  may  be  very 
difficult,  if  not  impossible,  to  give  any  meaningful 
interpretation  to. 

Once  one  has  decided  to  use  these  simple 
structures  one  might  wish  to  study  several  such 
structures.  The  question  arises,  how  does  one  compare 
different  factor  structures?  It  stands  to  reason  that 
a  perfect,  simple  population  structure  (all  ones  for 
loadings  with  each  variable  loading  on  a  single  factor) 
will  be  easier  to  detect  from  sample  data  than  a 
structure  with  low  factor  loadings  and  variables  which 
load  on  more  than  one  variable.  Here  the  second 
structure  could  be  said  to  be  more  complicated  than  the 
first.  It  would  be  desirable  to  have  an  index  number 
which  could  be  derived  from  a  given  structure.  This 
index  number  should  grow  in  magnitude  as  a  structure 
becomes  increasingly  complicated.  One  such  candidate 
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is  th«  average  uniqueness  of  the  structure.  Pennell 
(1968)  rejects  such  a  Measure  because  it  does  not  take 
into  account  the  fact  that  variables  of  equal 
coeeunality  may  load  on  differing  numbers  of  factors. 

A  given  factor  structure  is  said  to  be  more 
complicated  than  another  if  the  first  factor  structure 
is  harder  to  glean  from  experimental  data  than  the 
second.  The  following  is  a  proposed  index  for  the 
complexity  of  a  population  structure. 

Complexity  Index.  It  was  reasoned  that 
complications  in  a  given  structure  are  due  to  two 
components: 

1>  Complication  due  to  structure  —  if 
manisf estation  variables  load  on  single  variables  then 
a  simple  structure  exists.  As  the  manisf estation 
variables  load  significantly  on  more  than  one  factor 
the  complexity  of  the  structure  increases. 

2)  Complication  due  to  Uniqueness  -  if 
manisf estation  variables  demonstrate  high  communalities 
across  the  factors  (low  uniqueness)  then  there  is  a 
higher  chance  of  closely  reproducing  this  factor 
structure  than  that  of  another  structure  with  higher 
uniqueness. 


where  the  Aij  are  factor  loadings  in  the  ijth  position. 
N  is  the  number  of  factors,  and  M  is  the  number  of 
variables.  The  Hi  are  the  communalities  of  the  ith  row 
of  the  factor  structure  matrix. 

The  first  term  is  the  complexity  due  to  structure. 
The  second  term  is  the  complexity  of  the  structure  due 
to  uniqueness. 

The  second  term  is  simply  the  average  uniqueness  of 
the  structure.  As  the  average  uniqueness  grows  the 
complexity  grows.  This  term  is  bounded  by  0  and  1. 

The  first  term  is  the  quartimax  criterion  divided 
by  the  number  of  variables,  M.  The  quartimax  criterion 
is  minimized  at  0  when  perfect  simple  structure  is 
present.  As  variables  start  to  load  on  more  than  one 
factor  this  quantity  grows.  In  order  for  this  quantity 
to  be  useful  as  a  component  of  an  index  it  has  to  be 
bounded.  The  lower  bound  is  zero.  The  upper  bound  can 
be  found  if  the  following  maximization  problem  can  be 


solved 


MAXIMIZE 


M  N  J-l 


I  2  li  (aikaij: 

i=l  j=2  k=l  J 


SUCH  THAT 


1)  |ai  j|  s  l  Vi  =  1 , . . .  ,M,  Vj  =  1 

N 

2)  £  (a..)2  si  Vi  =  l . m 

k=l  1K 


,N 


M 

3)  £  aikaij  *  0  v*  '  3 

The  first  constraint  merely  requires  the  loadings 
to  be  less  than  or  equal  to  unity.  The  second 
constraint  requires  each  variable’s  communal ity  to  be 
less  than  or  equal  to  unity.  The  third  constraint 
reflects  the  mutual  orthogonality  of  the  factors. 

If  only  the  first  constraint  is  taken  to  be 
binding  an  upper  bound  of 


M* 


can  be  established  by  setting  all  the  elements  in  the 
matrix  to  1. 

If  He  require  both  the  first  and  second  constraint 
to  be  binding  thens 


N  J-l 


S  (a. ua.  . )  s 


i=l  j=2  k=l 


iki  j 
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and 


M  N  N 


M  N 


E  E  E  <alk)2  (a^)2  =  E  E  Uj,)2  E  (aiJt) 

i=l  j=2  k=l  1K  1J  i=l  j=2  ^  k=l  1K 


M  N 


M  N 


£  £  (aii)2  ^  (aik)2  s  51  S  (aij)2,1 

iTl  j=2  1J  k=l  lk  i=l  3=2  13 


similarly: 

M  N  M 

£  £  (a.,)2  •  1  s  £  1  *  1  =  M 

i=l  j=2  1J  i=l 

hence  M  also  acts  as  a  weak  upper  bound. 


This  author  was  not  able  to  determine  a  upper 
bound  with  the  third  constraint  binding. 

To  summarize,  the  above  index  is  submitted  as  a 
possible  candidate  to  compare  differing  structures  for 
inherent  complexity.  This  index  does  not  attempt  to 
account  for  the  possible  influence  due  to  the  ratio  of 
the  number  of  variables  to  the  number  of  factors.  In 
light  of  this  fact,  comparisons  in  this  report  using 
the  complexity  index  are  only  done  across  structures 
with  the  variable  to  factor  ratio  held  constant.  Two 
other  points  should  be  noted.  First,  a  weak  upper 
bound  is  used  to  normalize  the  structural  complexity 
term.  Undoubtedly  a  stronger  upper  bound  computed  with 
the  third  constraint  binding  would  lead  to  a  stronger 
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index.  Secondly,  it  mm  assumed  that  the  weights  on 
the  two  teres  were  equal.  This  assumption  implies  that 
complexity  is  equally  attributed  to  structure  and 
uniqueness.  If  the  index  proves  promising,  perhaps 
future  regression  studies  could  address  this  issue. 


Dimensionality  Assessment  Analysis 

The  two  dimensionality  assessment  techniques 
addressed  in  this  section  are  Kaiser's  criterion  and 
Catel 1 ’ s  scree  test . 

Kaiser's  criterion  is  to  merely  retain  all  factors 
whose  associated  eigenvalues  ae  greater  than  or  equal 
to  1.  Catell's  scree  test  is  a  graphical  technique  in 
which  an  investigator  looks  -for  a  break  in  a  plot  of 
rank  ordered  eigenvalues.  This  section  does  not 
attempt  to  make  statistical  statements  about  these 
techniques. 

Kaiser's  Criterion.  For  all  the  structures 
examined  in  this  study,  all  dimensionality  assessments 
based  on  Kaiser's  criterion  were  within  two  factors  of 
being  correct.  This  is  probably  attributed  to  the 
structurally  "clean"  sampling  populations  studied  and 
the  low  factor  to  variable  ratios  used.  Most 
dimensionality  assessments  were,  in  fact,  within  one 
factor  of  being  correct. 

Histograms  of  dimensionality  assessments  due  to 
Kaiser's  criterion  are  given  in  figure  6  for  structures 
1,3,  and  7.  The  correct  dimensionality  for  each 
structure  is  3.  Each  histogram  contains  a  total  of 
lOOO  dimensionality  assessments.  Note  that  the 
variability  in  the  dimensionality  assessments  is 
markedly  larger  in  structure  7  than  in  structure  3. 
Although  the  percentage,  difference  in  average 


****i^>  VJ.k  ,  v  <w 


communal i ty  is  only  about  14%,  the  percentage 
difference  in  the  complexity  index  is  48%.  No  solid 
conclusions  can  be  draxn  at  this  point,  but  at  least 
one  notices  that  the  complexity  index  is  moving  in  the 
right  direction. 

In  figure  7  histograms  are  presented  for  structure 
23.  This  structure  is  perturbed  by  3  nuisance 
variables  and  3  nuisance  factors.  It  is  one  of  the 
more  complicated  structures  stuidied  in  this  report. 
Although,  in  an  absolute  sense  there  are  6  inherent 
population  factors  to  this  structure,  3  of  these 
factors  account  for  less  than  3%  of  the  total  variance 
which  could  be  explained  by  this  structure.  As  can  be 
seen  in  the  histograms  a  dimensionality  assessment  of  3 
was  never  made.  Clearly  the  addition  of  nuisance 
factors  and  variables  can  impact  dimensionality 
assessments  via  Kaiser*s  criterion.  However,  for  the 
structures  studied  here,  one  can  expect  to  be  within  2 
factors  of  the  true  dimensionality.  To  this  author, 
Kaiser’s  criterion  seems  to  be  a  good  rule  for  thumb 
for  dimensionality  assessment. 

Catell’s  Scree  Test.  Since  this  test  is  graphical 
in  ntturBf  it  was  very  difficult  to  conceive  of  a 
method  to  apply  Mr  *-.e  Carlo  techniques  to  its  analysis. 
Clearly,  one  could  not  hope  to  examine  a  thousand 
graphs  visually  within  a  limited  time  period. 

Catell’s  scree  test  is  a  graphical  technique  used 
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to  visually  locate  the  hypothesized  break  in  ranked 
eigenvalue  magnitudes  which  should  occur  just  before 
that  eigenvalue  which  is  associated  with  the  correct 
dimensionality.  The  Scree  test  is  explained  in  the 
literature  review  section  of  this  thesis. 

Catell  would  have  an  investigator  retain  factors 
down  to  and  including  the  factor  which  begins  his  scree 
line.  To  test  this  procedure  the  following  approach 
was  taken.  If  the  scree  test  is  an  acceptable 
procedure  then  certainly  one  would  expect  the  method  to 
work  well  under  ideal  conditions.  An  ideal  condition 
for  an  investigator  would  occur  if  he  were  sampling 
from  a  population  like  structure  1.  Figures  8,9,10, 
and  11  are  Catell* s  scree  test  for  sample  sizes 
10,25,50,  and  100  respectively  for  structure  1.  Each 
plotted  point  is  the  mean  of  the  ith  ranked  eigenvalue 
over  1000  trials  at  the  particular  sample  size. 
Approximate  95%  confidence  intervals  are  provided  for 
the  means  of  the  eigenvalues  at  the  correct 
dimensionality  (3,  in  this  case)  and  1  plus  the  correct 
dimensionality.  Note  in  figure  8  that  there  is  no 
apparent  break  in  the  means  of  the  ranked  eigenvalues. 
This  situation  improves  markedly  as  the  sample  size 
increases,  figures  9,10,  and  11.  Notice  in  figure  11 
that  a  definite  break  in  magnitude  is  present  between 
eigenvalues  3  and  4.  Further,  notice  that  the 
confidence  intervals  for  the  two  eigenvalues  do  not 
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Figure  8.  Scree  Test,  Structure  1,  14*10 
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Figure  10.  Scree  Test,  Structure  1 
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overlap.  If  one  were  to  apply  Catell’s  scree  test  to 
the  means  of  these  eigenvalues,  clearly,  one  would 
retain  4  factors.  Thus,  when  sampling  is  accomplished 
under  even  ideal  conditions  Catell’s  test  has  yielded 
incorrect  results.  In  fairness  to  Catell,  however, 
figure  11  could  be  said  to  exhibit  what  Catell  refers 
to  as  a  double  scree  line.  Catell’s  procedure  is 
modified  when  a  double  scree  line  is  observed.  Factors 
are  retained  down  to  and  including  the  factor  which 
begins  the  upper  scree  line.  Under  this  modification 
the  correct  number  of  factors  would  be  retained. 

Notice  that  Kaiser’s  criterion  was  a  flawless  indicator 
for  structure  1  and  N=200.  Figure  12  is  another  ranked 
mean  eigenvalue  graph.  This  time  structure  23  provides 
the  data.  The  sample  size  is  100.  Structure  23  has  3 
nuisance  factors  and  3  nuisance  variables.  Notice  that 
the  same  break  occurs  between  the  mean  of  the 
eigenvalue  magnitudes  of  factor  numbers  3  and  4.  This 
time,  however,  the  confidence  intervals  are  quite  wide 
and  overlap.  It  is  not  clear  whether  or  not  a  break  in 
the  eigenvalues  will  even  appear  in  a  particular 
sample. 
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Figure  12.  Scree  Test,  Structure  23,  N=100 
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Factor  Interpretation  Analysis 

This  section  presents  the  results  of  a  regression 
study  undertaken  to  deteraine  if  the  saepling  errors  of 
the  experimental  region  were  predictable. 

Experimental  Procedure.  To  reiterate  the 
experimental  procedure,  first  sample  vectors  were 
generated  from  the  population  covariance  matrices. 

These  sample  vectors  were  then  used  to  form  a  sample 
correlation  matrix.  The  sample  correlation  matrix  was 
factor  analyzed  using  the  PCA  procedure  and  the 
resultant  factor  loadings  matrix  (of  the  correct 
dimensionality)  was  then  rotated,  via  a  least  squares 
procedure,  to  fit  the  original  population  structure. 

The  mean  square  discrepancy  between  the  sample  loadings 
matrix  and  the  population  loadings  matrix  was  then 
calculated.  This  mean  square  error  was  calculated 
across  all  the  loadings.  The  mean  square  error  (USE) 
is  calculated  by  the  formula 

N  M  -  2 

£  £<a u-V 

j-l  i*l 

M*N 

where  the  a.  •  are  the  factor  loadings  for  the 

^  J 

population  factor  loadings  matrix,  a.  •  are  the  factor 

^  J 

loadings  for  the  sample  factor  loadings  matrix,  h  is 
the  number  of  variables,  and  N  is  the  number  of 


-factors.  Ths  root  man  square  (RMS)  error  is  taken  as 
the  square  root  of  the  MSE. 

Performance  of  the  Complexity  Index.  Figure  13  is 
a  plot  of  MSE  versus  the  average  communal! ties  of  the 
original  structures.  Notice  how  structures  6  and  7 
produce  noticable  "bumps’  in  the  set  of  curves.  These 
two  structures  have  four  variables  which  load 
significantly  on  more  than  one  factor.  All  the  other 
structures  used  in  figure  13  contained  only  univocal 
variables.  One  would  expect  small  bumps  due  to 
sampling  fluctuations  but  the  aberration  due  to 
structure  7  seems  a  bit  severe.  Figure  14  is  a  plot  of 
MSE  versus  the  complexity  index.  This  graph  displays 
more  of  the  monotonicity  one  would  expect.  A  Bimiliar 
graph  is  presented  for  the  more  complicated  structures 
which  are  perturbed  by  nuisance  factors  and  variables. 
In  this  graph  one  notices  that  there  are  two  pairs  of 
structures  whose  complexity  indices  are  quite  close. 

In  all  but  one  case  the  corresponding  MSEs  were  quite 
close.  The  exception  occurs  for  structures  26  and  27. 
The  variance  of  these  MSE  values  are  of  the  order 
. OOOOl  and  so  it  seems  clear  this  particular  variation 
is  not  due  to  sampling  error.  It  is  probably  due  to 
one  of  the  complexity  index's  inherent  weaknesses  as 
mientioned  previously.  All  in  all  the  index  seems  to  be 
performing  fairly  well.  At  least  the  complexity  index 
is  an  improvement  over  using  average  communal ity  (or 


uniqueness)  as  a  criterion  of  a  population  structure's 
complexity. 

Regression  Study.  Several  different  regression 
models  were  hypothesized  and  tested  in  order  to 
determine  if  MSE  or  RMS  errors  could  be  reasonably 
predicted  as  functions  of  sample  size,  the  number  of 
variables,  the  number  of  inherent  factors,  complexity 
of  the  population  structure  and  the  interactions 
between  these.  The  condition  number  of  the  sample 
correlation  matrix  was  also  examined  for  its  possible 
aid  in  predicting  MSE  or  RMS  errors.  In  these  studies 
each  MSE  or  RMS  value  was  taken  as  the  grand  mean  of 
1000  iterations  on  a  particular  structure— sample  size 
combination.  The  same,  then,  is  true  for  each  sample 
condition  number.  Two  types  of  regression  models  were 
attempted. 

1)  Linear  models  with  interactions — these  models 
were  run  using  the  Statistical  Package  for  the  Social 
sciences  <SPSS)  (Nie,  1975).  A  stepwise  regression 
scheme  was  employed  for  variable  selection.  The 
following  models  were  run: 

a)  MSE  as  a  linear  function  of  sample  size, 
number  of  factors,  number  of  variables,  and  all  possibe 
multiplicative  interaction  combinations.  This  model 
was  also  ran  with  RMS  as  the  dependent  variable. 

b)  MSE  as  a  linear  function  of  sample  size, 
number  of  factors,  number  of  variables,  complexity 
index  of  the  population  structure,  and  all  possible 
multiplicative  interaction  combinations.  This  model 
was  also  ran  with  RMS  as  the  dependent  variable. 

c)  MSE  as  a  linear  function  of  sample  size, 
number  of  factors,  number  of  variables,  condition 
number  of  the  sample  correlation  matrix,  and  all 
possible  multiplicative  interaction  combinations.  This 
model  was  also  ran  with  RMS  as  the  dependent  variable. 

2)  Nonlinear  models — These  models  were  also  run 
on  SPSS.  Nonlinear  production  functions  of  the 
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Cobb— Douglas  type  (Nicholson,  197B)  m ere  run  over 
various  combinations.  The  Cobb-Douglas  type  function 
was  chosen  because  preceived  nonlinearities  which  were 
"eyeballed”  in  the  data.  Also,  the  Cobb-Douglas 
function  is  flexible  in  the  sense  that  it  can  mold 
itself  to  many  different  shapes.  The  Cobb-Dougl as 
function  is  of  the  form 


I  =  B0  X®' 


The  -following  models  were  runs 

a)  RMS  with  independent  variables}  sample 
size,  number  o-f  -factors,  and  number  o-f  variables. 

b)  RMS  with  independent  variables}  sample 
size,  number  o-f  -factors,  number  o-f  variables,  and 
complexity  index  of  the  population  structure. 

c)  RMS  with  independent  variables:  sample 
size,  number  of  factors,  number  of  variables,  and 
condition  number  of  the  sample  correlation  matrix. 

The  following  are  used  as  abbreviations: 

1)  Sample  size — N 

2)  Number  of  factors — FAC  or  F 

3)  Number  of  variables — VAR  or  V 

4)  Complexity  XhdQx  — C 

5)  Condition  numb&f - K 

6)  Interactions — -an  ev&osple  is  NxF  or  sample 
size  *  number  of  factors 

Figure  16  is  a  tabular  comparison  of  the  results 
from  the  linear  models.  Note  that  the  best  predictions 
are  made  from  the  model  which  includes  the  mean 
condition  number  of  the  sample  correlation  matrices. 
Note  that  sample  size  is  the  most  significant 
independent  variable  in  all  the  models.  It  is 
interesting  to  note  that  the  addition  of  the  complexity 
index  into  the  first  two  models,  although  only 
improving  the  model’s  predictability  slightly,  creates 
a  situation  wherein  the  second  most  significant 
independent  variable  is  a  interaction  term  on  the 


complexity  index.  The  coefficients  of  the  predicitive 
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aquations  given  by  the  6  Models  are  given  in  figure  17. 
Care  should  be  taken  when  atteepting  to  predict  froa 
regression  relationships  which  use  the  condition 
number .  If  the  sample  size  is  less  than  23,  reasonable 
results  can  not  be  guaranteed.  The  variability  of  the 
sample  condition  number  in  the  region  studied  was  of 
the  order  1.0E+15  for  the  sample  sizes  of  10.  In 
summary,  RMS  errors  are  more  accurately  predicted  than 
MSE.  Taken  in  pairs,  the  standard  errors  of  the 
estimates  when  normalized  by  their  respective  mean 
estimates  are  always  lower  for  RMS  regressions  than  for 
MSE  regressions. 

Figure  18.  is  a  tabular  comparison  of  the  results 
from  the  logl inear  models.  Note  that  these  models 
display  slightly  high  adjusted  r — squared  values.  Here 
again,  notice  that  the  sample  size  is  the  most 
significant  independent  v?~iable.  In  the  second  model 
complexity  is  the  second  most  significant  independent 
variable.  The  nonlinear  models  are  slightly  superior 
to  their  linear  counterparts,  the  standard  errors 
normalized  by  the  log  of  the  mean  estimate  are  in  all 
cases  lower  than  the  linear  models. 

The  regression  study  shows  that  for  the 
experimental  region  studied  the  errors  due  to  sampling 
in  a  factor  loadings  matrix  can  be  reasonably  predicted 
by  either  linear  models  or  nonlinear  models. 
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Conclusions 


A  limited  examination  of  Kaiser's  criterion  and 
Catoll*s  seres  test  indicates  that  Kaiser's  criterion 
is  usually  good  to  within  a  -factor  and  was  always,  for 
the  structures  and  sample  sizes  addressed  in  this 
report,  no  more  than  3  factors  from  the  true 
dimensionality.  In  an  ideal  sampling  situation 
Catel 1 ' s  scree  test  seems  to  retai n  one  too  many 
factors,  and  it  is  not  always  easy  to  identify  the 
scree  line. 

The  concept  of  a  complexity  index  appears  to  be 
promising.  If  possible,  a  stronger  upper  bound  needs 
to  be  found  for  the  first  term  of  the  index.  The 
possibility  of  unequal  weights  for  the  two  terms  could 
be  investigated  through  some  further  regression 
studies. 

The  results  of  this  research  indicate  that  it  is 
reasonable  to  estimate  an  overall  mean  error  due  to 
sampling  for  structures  in  the  particular  experimental 
region  addressed  by  this  report. 

This  author  believes  that  this  report  has 
demonstrated  that  sample  factor  loadings  matrices  are 
sensitive  to  sample  size  and,  more  importantly,  the 
structural  complexity  of  a  given  population  factor 
loadings  structure. 

The  author  recommends  future  research  which  would 


6? 


address  the  stapling  distribution  of  the  coeplexity 
index. 
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